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IBSTRACT .« 

Many instruments designed to assess reading 
sompreheusion reflect instructional tasks that have .not clearly been 
shown to b€ a part of the process of comprehension. If the 
eeasufeaent instruments are to have construct validity,, however, they 
mst be created to reflect what we know .about cognitive processing, 
iecent work in cognitive psychology has begun to aid us in codifying' 
:cme of the basic factors or elements of comprehension. Three 
>articular areas of research in cognitive psychology are explored as 
potential sources of items and activitiies for Assessing reading 
zomprshension: imagery- and associational aids in recall; network 
:heory relating classes and examples of concepts; and "scripts" that 
rontain information about situations encountered with regularity. 
(AA) * 
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* / V I 

Processes and Tasks of Reading Comprehension 4 * j 

; " ; ; 7 ' ' " V '. 

In order to develop perspectives/ regarding thfiw assessment Of reading 
comprehension, one must develop satisfactory definitions of both the processes 

and tasks involved for a reader to comprehend written discourse. Defini- k 

» t - < 

fcions are vital when discussing comprehension. It becomes crucial to un- < 

derstand what goes on inside the reader's head and how that relates; to 

assessment and instruction of reading comprehension. * 

, / ' ; 

What $ve the probable processes involved in reading comprehension? . I 

. 'can begin by suggesting that the psychological processes of reading com- 

prehension are similar to those involved in all other coxmnunixiations * 

processes. That is, whet^fr listening, speaking, reading or writing, we 

assume that comprehensicfo is a complex series of cognitive and' language 

factors operating in tandem that .assist humans in gaining minings during 

communication. These interrelationships aid in obtaining meaning. Com- 

prehension, then, is the acquisition of meaning as a result of some form 

«of, language communication. These meanings occur as the individual is 

< ** ? ■ 

aware of, receives, perceives, processes, stores and/or retrieves infor- 

' *• .* t«. . * 

nj^tlon from external stimulation. Comprehension is the process of in-* ^ 
ternalizing external events or stimuli for the purposes of obtaining 
meaning. ^ < * 



Presented at the Annual Convention, of the Internation Reading Association, 
May, 1977. 



Reading tasks, on the other hand, are, the skills and/or activities 
th^^make up the' instructional aspects of comprehension. They are the 
things we ask studtents to complete i\n an attempt to leafn to read. They 
may tfe worksheets, workbook materials, discussion questions, fill-in-the- 
blanl^exercises, and other forms of materials that reflect the instructional 
components of reading. We refer to these tasks as the content of reading. 
But by themselves , they do not make'up the processes of reading comprehen- 
sion. Often the difficulties associated with assessment of reading compre- 
hension come £bout because we assume the cognitive processes and the tasks 
we have generated to teach reading comprehension are one and the same.. 
This -is often not the case in the assessment of reading comprehension. 

Many comprehension instruments reflect instructional tasks that haye 
not clearly been shown to be part of the processes of comprehension. For 
example, suppose we ask students to identify the main idea of a passage, 
locate its significant details or read the passage critically. After 
reading the passage, the -student is directed to answer several questions. 
We provide the questions and then ask the student to choose, from- alter- 
native answers, the one statement that provides the best answer. Have 
we really defined the main idea, significant details or critical reading 
before asking the student to locate answers to our questions? That is, % 
do our own labels insure that something is what we call it? As an aside, 

isn't the student's own- definition of thp main idea an<J the significant 

»■■-■ \ • . ■ * 

details and his/her answer to thkt question more repealing of the cognitive 

. » - ; ■ , ' . * ; r 

processes used to comprehend the passage? ' y 

Another situation becomes^ apparent when we examine most standardized 
measures of reading comprehension. The items are created; to distribute 



3 

student scores along a continuum of good to poor performance;, Thuu , l.lc 
items that are finally used in the assessment instrument discriminate 
among students taking the test. These items may be later revised so that 
only a few students get the correct score. The correct response, while it 
may be a useful index of student performance in relation to (his/her peers, 
may or may not reflect the student f s' level of cognitive processing. Choosing 
the correct, response may reflect the student 1 ? ability to'detect the item - 
writer's level of cognitive processing. This simply means that student 
scores on standardized a tests of reading comprehension reflect but one 
aspect of written language comprehension. It further suggests than aii 
7 adequate assessment of reading comprehension must include both standard- 
ized and teacher-made instruments. Teacher-made assessments include both 
structured and nonstructured activities, Thus, the kinds of questions 

teachers write 'an4 those they ask to assess reading comprehension become 
\ 

a vital part of assessment. w 

The ideal measurement instrument of reading comprehension, I submit, / 

/ 

is one that purposely sets about assessing reading tasks that have been 
created to reflect what we know about cognitive processing.' Thus, the 
critical tissue in creating reading comprehension instruments is construct ' • 
validity. That is, do the tasks reflect what research has deinoAstrated 
to be the underlying traits or processes of comprehension? 

Recent work in cognitive psychology- has begun to aid us in codifying 

9 

some of the basic factors or elements of comprehension. -/As Anderson 
suggests, / 

/ 

We now have^a conception of the processes involved in learning 

j " N 

from written discourse which, while tentative and incomplete, 

l ' . 4 

v • / * 



provides a useful beginning. \ . . Elements of text are first 
encoded in terms of perceptual features , Since the relevant 
perceptual features of text are orthographic, this can be 
called orthographic, encoding . The next level of processing 
probably involves acoustic, features.* At this stage, a which 
can be "called phonologic all encoding , strings of words are" 
rendered into implicit (ot explicit) speech,. Finally, therfc 
may be semantic encoding , * that is, the person may bring to 
mind meaningful ireptesentaticfti baied on the" words ha sees,* or 
hears himself saying • %0f course** a person must be able to 
coordinate the "surface information 1 ? embodied in" the ortho- 
graphic and phonological codes in tferms of linguistic rules^. 
(Anderson, pp. 145-6, 1972) , . , . 
, Let us look at some of the elements or factors that se^ta to, reflect 

the underlying processes involved in reading comprehension, 

* * ' ■ - t ■ 

I wi^jh to, pose three possible areas chosen from ^research in cognitive 

psychology that have potential as sources for items and/or ^activities that 

assess reading comprehension. These sources are not s'een as "mutually ex- 

J 

elusive or statistically independent levels. They are more Chan likely 
highly interrelated. Th^y reflect recent research efforts. in semantic 
- encoding and retrieval, and as such could be used appropriately for the 
selection of various types of items. Following a brief description of 
each of the areas will be examples chosen from standardized tests that 
attempt to illustrate the levels. Also informal questioning strategies 
and' suggestions for assessment will be made. ., , 



.5 





Grammatical Propositions 


Levels 


of Representation 






Images . « 




Networks 




Scripts 




Figure 


1. Semantic Encoding Processes of Comprehension 



Grammatical Propositions in Comprehension 

We know that the meaning of a sentence or passage is more than- the 
sum total of the separate meanings of the words used to express the idea. 
How are words interrelated to communicate meanings involving a sj0&Les of 



concepts? The answer to this question has been suggested by both Chomsky 
(1967) and Fillmore (1968). These writers, and others, have developed a 
system of rules, or propositions that suggest that we automatically pre- 
dict hbw strings of words are interrelated for the purposes of obtaining 
meaning. 

Grammatical interrelationships are described as a system of rules 

*' . • * • ■ 

governing relationships among and between words that aid in transforming 

deep -structure (meaning structure) to surface structure. Tlie.se rules are 

v 

overlearned and automatic. We rarely think of them unless we perceive, 
some interference that hinders meaning. Thus in the sentence: Dogs, 
scare cats, we* give no thought to the orci^r in which these words appear 
on the page. If the sentence appears: Cats dogs scare, we stop and 
ponder seme possible meanings \ When describing language, it is appro- 
priate to distinguish between the words that are used to express spoken 
or written ideas and the semantic representation of those ideas. Words 

- • 4 ' . ' 6 



■ k ■ - ■■ ; 

that ;are comprised of speech sounds or their written counterpart's repre- 

L V 
sent the surface of language. Surface structure may parallel orthographic 

and phonological encoding that Anderson referred to earlier*;* That is, 

sentence o^der is only important in communication when it is perceived as' 

disorder. Words-, then, are- used to ; express ' inner thought .and meaning. , 

Thus, we wish to know whether or not people ^parr-perceive the deep or 

meaning structure of sentences. Since the^ormal student enters the'edu- 

cational system in command of the rules that translate -surface structure 

> 

to deep, structure, and since, as Anderson point's out, thetee are separate 

' • .i V , ^ . . ' / . 

from the semantic encoding processes involved in comprehension, we miould 
assume 4 that the grammatical processes are working adequately and need not 

.y . • * ' / . ■ 

be assessed. ^ - ^ 

Representation in Comprehension and Tasks for Assessment: Imagery 

It is at the level of 1 semantic encoding^fco which Anderson refers, 
that reading comprehension assessment must take place. Several writers 
(Bruner; Neiser, 1967; $ticht, 1975) have 'suggested that recall of infor- 
•mat ion is aided by imagery and associational phenonjfcna. Thus, a mental 
representation of a previously learned Or encountered concept or event 
occurs when we recall that event. It is suggested that w& have mental' 

'"pictures" or traces of concepts stored in memory. Thus, Anderson (1972)-% 

• » * * * 

cited two typeg of recall phenomena? theory : images and semantic features. 

Image theory suggests^^h'at, when presented with a verbal stimulus, f\x 



dividuals ,are able to reconstruct mental pictures of concepts.^ This .1.8 
trug to th6 extent that the concept has/picturable properties . Co* 
qiiently, abstract concepts would be less v amenable to image reconstruction 
than concrete, readily defined concepts. Hence it is easier to Reconstruct 




• / ' / \ * ■ ' 

an imag^e* of l ^dog tt than "republican form of 'government l n -Stithl (i L )/b) C\ 
' reefers to tjii'a phenomenon. as an iconic mode of representation.' He ex- , V 
k tends this notion to suggest that we have rtgtftal programs stored in memory 
J that allow' us to externalize certain conceptS*by drawing pictures . Thus,.. i \ 
an initial level 0f assessment reflecting cognitive processes would involve 
tasks that emphasize 'pictdgraphic .stimuli or responses. It ,would be appro- 
' priate to use pictures to represent concepts. Concepts would 'be repre- ^ \ 

■ > . ^2 . ■ ' \ 

Bunted by words. After reading or listening to words >or sentences stu- r J 1 ,,x 

' • • . 7 . A 

dents might Se asked-to tdentij^y^a picture that raprtesents the concept. ^ «7*y 
The following types of items illustrate this level o£ semantic , encoding •< * 

V — - • ■ . v J • 



Insert Figure 2 / ' * * ' ■ • «/ 



Network&Representations fr , ^ * v ^ 



Network theorists -(Collins and Quillian, 1969; Lind^a^. and Norman,^ 
1972) provide datA^ind±cating that semantic encoding and retrieval ^e- 
suit from stored concepts and their attributes. They state that we bring 
to consciousness not only^the stored concepts but their accoirfjpanyinK at- ✓ 
tributes • 'These concjepts are stored in an hierarchical ordefc an4 cate- 
gorized by unique features or attributes.. Thus, things are ordered, in. 
memory, according to their membership 'in classes'. We relate classes f->y 
i examples and Categorize them on the. basis of their physical and/or func- 



tional features. Concept storage and rfctrievalris also aided by specified 



examples of e^ch concept held in memory. Concepts, then, are categorized 
on the basis &$\ their attributes, into which class they fall, and by 



specific examples. When individuals are presented with stimuli, either 
verbal or written, they jperform a memory search to locate previously 

8 : 



Figure 2 

EXAMPLES OF IMA^B. A'h^KESbKlAT.iUJ^' ASSESSMENT 



Mark an X bn the hat. 





< 



(Boehm, 1969)' 



.■ \ 



Circle the shoe. 








I 



N 



(Lorge, Thojrndike, & Hagen 1964) 



Circle the picture that shovje raining. ~J 

• . • _ ^ — 7 -V 
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g (Lorjp, Thorndike, & Hagen, ^^64) 



• encountered elates,* examples, pr features), lli^ assoc. tat tonal procefls 

^then) becomes "a critical aspect of ££>mprehension.' Variouef experimental 
*<• setting* have .been utilized td ve/ify semantic searching processes. Word 

• ^association tasks and verification 1 of responses to prepositional statements 

• >' s " / : j 7 

by answering >Ather true o*r false have lieen' utilized to confirm network / 

* theories. # Thus* an assessment of r^adifig comprehension utilizing^ tasks 

* fromv^e^work theory would "include tasks that involvej^verif i\:ationv of con- 
ceptual hierarchical structures. It would further include tasks that in- 
* ^61ye identifying classes and examples of concepts • The nttmber of fea- 

3^ res a n individual could identify and the time required to identify 

thoS^features would be'two critical dependent variables* in the assessment 

f • 

... . 'o ^ 

pi reading comprehension. The following examples illustrate this point: 



Insert Figures 3 &. 4 

\ ' - ' "~ ' = : ~ * " 

Scripts * 

Several writers have advanced' tl\e level of network theory to encom- 
pass interrelationships among ^concepts (Lehnert, 1975; Pearson and Nichol- 

t 

son, 1976). Theses writers contend that scripts are useful model's to de- 
scribe human memory organizations. Scripts are defined as memory units 

.Containing information about situations that are encountered with regu- 
larity. They are expectations about everyday happenings. We have so much 
experience with thes.e happenings that we take, for granted predict what 

u will occur. Thus, when you attend; class , ydu have internalized your ex- 

pectatifrns. When you later read .about situations similar to attending 

y 

class, yoii are able to make accurate predictions regarding outcomes . 
* ■ » » ■ 

t is*, your "attending class, script" is a valuable asset in predicting 

' 10 



/ 



EXAMPLES OF NETWORK REPRESENTATION ASSESSMENT PICTORI ALLY PRESENTED 



r" 



Which animal is huge? 



o 





o 



(Karlsen, Madden, & Gardner* 1975) 



Which thing has been woven? 





(Karlsen, Madden, & Gardner, 1975) 



A IThe ball is big. 




( 



O • ' 

(Karlsen, Madden, j & Gardner, 1975) 
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turn 

: o 



;i ■ cjimb 



whok 
■ Oi 



fence 



< ^ince 
>^ o 

over w / 



clip 

• , p • 

feit 
o 

other 
„0 j 



/ 
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(Karlserir Madden \ & Gardner, 1975) 



gXAMtHJSS OF- NETWORK REPRESENTATION ASSESSMENT IN WR ITTEN DISCOURSE 
r -~x — * 



When two things are much 
the same, they are, 

14 Jfc alike alone 



iarjye 

o 



To q iqgle means to 

16 chatter joke 

o ' 'o 



laugh 



(Karlsenv Madden, fit-Gardner, 1975) 



\ 



-Not just anybody is permitted to drive a huge 
semi-trailer- trUck long distances. Drivers must 
have healthy bodies as well as keen sight ancl 
hearing. They must know a lot about the truck, 
thedriving laws of many states, and hoiv to drive 
in different kinds of weather. They learn these 
competencies iiyspeciaj 4 school$. ' • 

Drivers alio learn r wlvut todoih an emergency. 
They carry fiVc^xtiriguishers and first-aid kits, 
as well as flashlights flares, and tanterns. 



1 1 . Competencies are 

A. duties. 

B. ideas.. 
CD skills. 

D. tricks.,. 



12. Flares are ■ 

A. bandages. 

B. road maps. r 
' Cv stretchers. 

warning lights. 



(Hanna, Schell / & Schreiner,, 1977) 



1. House 



<25 



home 

B. live 

C. mother 

D. water 
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9- Difficult ' v. - 1 

A. bad ' . 

CS> 'hard 
C alone 
, D, - 'regular 

(Hanna, Schell, & Schreiner, 1977) 



. , \ . ' • ' • * . • * 

meanings, .You expect the instructor to arrive with notes in hand, take 
roll, deliver the lecture £nti ask questions . As in the case of grammat.r 
icaJL surface structure situations, you are only surprised when these 
. events ^do not^occur in theit predictable sequences. Further, scripts are 
acquired through extensive cognitive experiences • You generalize your 

expectations about situations to the limit of the experiences - you have had 

\ • - 

" previously. { * ' • 

* Some efforts have been made using scripts 4 in sto^y grammar to deter- 
mine meanings of^olktales and myths* (Ma^dler and Johnson, 1977). They 
su&gept that in stories representing the oral tradition, such as folk- 
taleisi' subjects have predispositions about - the internal* structure which 
facilitate both encoding and retrieval. The term "story schema 11 refers 
" to the set of expectations -about internal structure. Story schemata come 
^om two sources: Listening to many stories and internalizing knowledge 
about the sequencing of events within a story. The knowledge consists of 
typical beginnings (settings), plot, roles^of antagonists, protagonists,, 
conflict resolution, and endings. The other source comes from general 
"Heaowledgf about causal relations and various kinds of action sequences. 
It is suggested that only those perceptions, feelings, actions and events 
which have. to do with the ongoing plot or story line are represented in 
the story schema. Thus, encoding is aided by story specific content. 
Ma^dler^ and" Johnson describe a story schema as follows: 
During encoding, the schema acts as a general framework within 
which detailed comprehension processes take place. This frame- , 
work performs several functions. First, it directs attention 
to certain aspects of the incoming material. For example, 



! tatemen ts in the setting of a folktale . . -. are always 
' * Jr ^ lev ant t0 i^er events; they warn the listener that cer- 
m t t • t a i^ fact s should be kept in min^ SecondV the' framework helps 
iSC ^ner keep' track of has gone before. .It provides 

a summ ary tfc increases the predictability of what will im- 
mediately follow. Third, the framework tells tlie listener' 
hen Some part of the 8tory is complete -and can therefore be 
t?reci > °* is incomplete and therefore must be held until 

m ° re ^at^ial has been enco4ed. (Mandler and Johnson, p. 112, 
\ l9 /7) / 

8 plication ought to invoke, in any English teacher's "script," 

& sense p£ s 

rea 8surance. This research provides some satisfaction in 

at ^hen teaching literary devices we were^e^gaging in semantic 

^ncodi nR 

ia retrieval activities! Data from script' and schema' research 
^serui in establishing stahdards of construct validity in the 
or reading comprehension. 

* ct *e following , examples illustrate items that reflect the 
.* CTX?t ^P^sentatibnal level: 



Insert Figures 5 & 6 

The 



0 



candar^ 



examples of various types of test items were selected from * 
iz Sri 

u me asures of reading comprehension. There are, of course, 



^veral act 

lvlt tes that can be used as informal reading compretu 



ens ion 

iques 



Xma Se level representation responses could be provided by stu- 
Verbal Messages could easily be pictured by students and the 

0 

14 t 



* Figure. 5 - 
m m p j , - t* 

EXAMPLES OF SCRIPT REPRESENTATION ASSESSMENT. PICTORIALLY PRESENTED 



( 



J 




bark 

. o 

store 
carry 



(Karlsen, Madden, & £argner, 1975) 




Circle the picture in which the ladyH^ jiot in the gard 



en, 






f 

- t 



(Lorge, Thorndike, & Hagen, 1964) 




EXAMPLES OF SCRIPT REPRESENTATION ASSESSMENT IN WRITTEN DISCOURSE 



/ 



For breakfast Pat had eggs, milk, and toast. 
Her brother Alan had corn flakes and a roll. 
Mother drank coffee aud ate a roll, father hiad 
already eaten and l/ft. / 



2. What time of day was it? 

' ChS^ Morning 
jp B. Noon V 

C. Afternoon 

D. Night 



5. Who is Pat? 

, A, The brother 

CD^Tf^4^ter 
U C. TWe father 
D. The mother* 



(Hantra; Schell, & Schj'einer, 1977) 




^ pencils kept falling to the floor. 
DaffifpgTspelling paper turned up in Lisa's reader. 
One day the clock would be fast, the next it ' 
would be slow. Twice Mr. Garcia's chair was on 
his desk. Sara found both Susan's and Lee's 
scissors in a library book. And once ^hen every- 
tfcfdy came in from recess on the playground, 
M BOOr was written on the board. Yet nobody 
ever saw the prankster . Could it have been a ghost? 



26. This sto)y probably takes place 
at church school. 
X^jr <jS) in a schoolroom. 
r ■ C. in the library. 

D. on the playground. 



\ 



3l ^Wh'ch title test describes this paragraph? I ; ' 
C^J ThcUnseen Prankster 

B. Halloween Happenings ' 
£. Lost and Found . / 
D. Danny and the Ghost 

i 

(Hanna, Schell, & Schreiner, 1977), 
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responses be construed as representations of semantic encoding processes* 

? • • ' - 

While these "messages" may not be too elegant, they could be an indica- 
tion of an individual's leVejL of comprehension. 

0 > 
Several questioning strategies could easily be generated to assess 

network levels of semantic encoding or retrieval* Students could be asked 

to provide attributes of concepts as written responses* Activities could 

% ' 
be devised whereby teachers provide attributes and students infer the con- 
cept from a list of critical classes, features or examples* .Any combina- 
tion of arrangements coulct^be developed with these activities. The cloz^ 
technique appears to be a rich resource for items and activities that 
assess network processing. In all instances students are required to 
provide missing elements in^jnessages . These elements can be systemat- 
ically ^deleted to illustrate interrelationships among concepts. The cloze 
approach would also be highly useful in assessing Student's [perceptions 

■ " 

of script representations. a 

* * 



References 

^Anderson, R. C. How to construct achievement tests to assess comprehension. 
Review of Education Research . 1972, 42, 145-6. 

j* ^1 ^ n - r, J# S# 13x6 course of cognitive growth. American Psychologist , 1964, 
19, 1-15. . - 

: Chomsky, N. The formal nature of language. In E. H. Lenneberg (Ed.), 

Biological fouhdat ions of language . New York: \ Wiley," 1967; 

Collins, A., & Quillian, M. R. Retrieval time from semantic memory. 

* » . " ' ' * ( • ■ , 

Journal of Verbal Learning & Learning Behavior . 1969, 8, 240-247. ' 

Fillmore, 1 C. J. The case for- case, in E/ Bach & ft. G. Harms (Eds.), 
Universals in linguistic theory . . New York; ■ Holt, Rinehart & Winston, 
1968. ' ' , '" ' 

Lehnert, W. Question answering in - story understanding system . Research 

Report No. 57, Advanced Resfearch Projects Agency, Department of De- 

. >i 
fense, Office of Navai Research, Contract N00014-75-O1111, 1975. 

i. 

Lindaay* P. H., & Norman, D. A. Human information processing . New 

York: Academic Press, J.972. . ' 

Mandler, J. H. , & Johnson, N. S. Remembrance of things parsed:; Story 

structure and recall. Cognitive Psychology , 1977, 9, 111-151. 
Neisser, U. Cogni t ive psycho logy . New York: Appleton, 1967. 
Pearson, P. D., & Nicholson, T. Scripts , texts and questions . 

Paper presented at the NRC Convention, Atlanta, December, 1976. 
Sticht, T. G. The Acquisition of literacy by children and adults . 

F^er presented to . the Second Delaware Symposium on Curriculum, 

Inatri^tion, ai^l Learning: jThe Acquisition $of Reading, University 

of Delaware, June, 1975. 

: '. • 18 \ 



, Test 8 



York: . Psychological 



> if 1 

Boehm, A. E. Boehm Test of Basic Concepts , 

Corporation, 1969. 

Hanna, G., Schell/L. M., & Schreiner, R« The Nelson Reading Skills 

Test, (levels A, B, and C) / New York; ^oughton-&ffliri, 1977. 

• ■ \ * * ' • ' ' ' . - 

Karlsen, B., Madden, R.£ & Gardner, E. F. Stanford Diagnostic Reading 



£egjt, (MRC Edition) . N£v York: Har court, Br aSqe, Jovanovich, Inc^, 
• 1975. x : / \ . t . ■ . , • 

Lorge, I., Thorndike, L., & Hag en, E. The Lorge-Thorndike Intelli - 
gence New York:' Houghton-Mifflin, 1964. ■ * 



r 



/ 



V 



• . - * 

19 



